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Abstract 

We establish conditions for uniform r-th moment bound of certain M. d - valued 

functions of a discrete-time stochastic process taking values in a general met- 

^ • ric space. The conditions include an appropriate negative drift together with 

^ ■ a uniform L p bound on the jumps of the process for p > r + 1. Applica- 

f-^. ! tions of the result are given in connection to iterated function systems and 

biochemical reaction networks. 
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1. Introduction 

Stability is an important property in any form of dynamical systems. For 
deterministic dynamical systems, stability is mainly concerned with different 
types of behavior of the trajectories of the system which start near the equi- 
librium point. For the stochastic counterpart, many notions of stability have 
been developed in the context of Markov chains or more generally Markov 
processes. Typically, the study of stability of a Markov chain involves check- 
ing the existence of invariant measures and investigating various types of 
convergence of the transition kernels to the invariant measure. Further inves- 
tigation involves seeking criteria for ergodicity, Harris recurrence or positive 
Harris recurrence. While different types of Lyapunov techniques are used for 
studying stability in the deterministic case, the corresponding investigation 
for Markov chains is carried out by suitable uses of Foster-Lyapunov func- 
tions. The essence of the matter is the following: given a process {X n } ne z+ 
taking values in a Polish space U, one constructs a non-negative measurable 
function V : U — > [0, oo), called a Foster-Lyapunov function, such that the 
process {V(X n )} n£ i + possesses certain desirable properties, e.g, some kind 
of Foster-Lyapunov drift condition. The process {V(X n )} ne z+, being real- 
valued and nonnegative, often admits easier analysis and standard results 
yield various conclusions about recurrence, ergodicity or rate of convergence 
of measures, etc, for the original process {X n } n€ z + . A good reference for 
various Foster-Lyapunov drift conditions for discrete time Markov chains is 
[1]. For various results concerning invariant measures of Markov chains, see 
[2] and [3] for general Markov-Feller operators. For continuous time Markov 
processes, [4] and [5] discuss various techniques for checking stochastic sta- 
bility. 

In this paper we consider a different notion of stability, namely, uniform 
moment bounds for multi dimensional functions of discrete time stochastic 
processes. More precisely, given a stochastic process {X n } n( zz + taking values 
in a metric space U and a sequence of functions {G n : U — > M. d }, conditions 
are sought such that sup n E[||G n (X n )|| r ] < oo. Uniform moment bounds of 
stochastic processes or functions of stochastic processes have important ap- 
plications in several disciplines like queueing theory, control theory, physics, 
etc. For an M-valued process {X n }, Pemantle and Rosenthal [6] established 
conditions for sup n E[(X+) r ] to be finite. The conditions involve a "constant" 
negative drift together with a uniform L p bound on the jumps of the pro- 
cess for p > r + 1. The fact that the result does not require existence of 



Lyapunov functions makes it particularly useful, as explicit construction of 
suitable Lyapunov functions is often a difficult task [7]. For a Markov chain 
{X n } taking values in a general metric space, [8] used the theory of excur- 
sions of Markov processes to establish a uniform L^ bound on an R-valued 
function of X n . Their hypotheses require the existence of a certain derived 
supermartingale with a prescribed rate of decay when the process stays out- 
side a compact set. While this approach does not work directly with drift 
conditions as in the Foster-Lyapunov function approach, the existence of the 
desired supermartingale is in general not straightforward to verify. 

Our paper generalizes the one-dimensional result of [6] in two directions: 
first, we consider IR^-valued functions of the stochastic process {X n } taking 
values in a general metric space; second, the drift condition is generalized to 
incorporate a number of scenarios. More precisely, our main theorem reads 
as follows: 

Theorem 1.1. Let {ft, J 7 , {J-^}, P) be a filtered probability space, U a 
complete and separable metric space and C C U (Borel) measurable. Let 
{G n : U — > R d } and {H n : U — > R d } be sequences of measurable functions 
satisfying 

(i) for every n, G n , H n : U \C -»■ W l + ; 
(ii) for every n, G~ l G n {U \C) = U\C; 

(iii) there exist constants a, b > 0, such that a ^ inf n inf x ^c' 1 1 -ff n (a;) 1 1 , 

sup n sup x £ C ||^n(x)|| ^ b and sup n sup a;eC ||G n (x)|| ^ b. 

Let {X n } be a sequence of { J-^} -adapted U-valued random variables. Assume 
that Xq G C and the following two conditions hold: 

(iv) for all n ^ 0, 

(1.1) E[G n+1 (X n+1 ) - G n (X n ) | T n ] ^ -H n {X n ) on {X n $ C}; 

(v) there exist constants L > and p > 2 such that for all n ^ 

(1.2) E[\\G n+1 {X n+1 ) - G n (X n )\Y | F n ] < L. 

Then for any < r < p — 1, there exists a constant n = n(p, a, b, d, L, r) such 
that 

sup E[||G n (X n )|| r ] <77. 

nGZ + 



If in (v), instead of (1.2) we have 

(1.3) E[\\G n+l (X n+l ) - G n (X n )\\P] < L, 

tlien for any < r < p/2 — 1, t-here exists a constant n = n(p, a, b, d, L, r) 
such that 

supE[||G n (X n )|r] ^r). 

nSZ + 

Here, N = {1, 2, . . .}, Z+ = {0} U N, R d + = {x e R d : x t ^ 0, i = 1, . . . , d} 
and ||-|| denotes the Euclidean norm on R^ . Typically in many applications, 
G n = G is a continuous function and C C [/ is compact. Therefore the 
condition sup n sup^,^ ||G n (x)|| < b automatically holds. In fact, Theorem 
1.1 also holds if the condition sup n sup xGC ||G n (x)|| < b is replaced by the 
condition sup n E[||G n (X n )|| p l {XnGC . } ] < oo. 

Next, note that, since G~ 1 G n (f/\C) D U\C, (ii) is equivalent to requiring 
G^GniJJ \ C) C U \ C. A necessary and sufficient condition for (ii) is 
G n (U\C)nG n (C) = (D. 

The inequalities between the various vectors in Theorem 1.1 are inter- 
preted component-wise, i.e., for i,i/6 M d , we have x ^ y if Xj ^ t/i for all 
i — 1, . . . , d. One salient point to note in Theorem 1.1 is that no Markovian 
assumption on the process {X n } is made. The component- wise inequality 
used in (1.1) is the natural partial order in the first orthant M^. However, 
Ml plays no special role in the proof of Theorem 1.1 and the result for a gen- 
eral orthant is stated in Theorem 2.7 with the partial order of M^_ replaced 
by an appropriate partial order of the orthant considered. 

Our result can be particularly helpful in queueing theory, control theory 
where a uniform bound on the variance of the states of a multi-dimensional 
stochastic system is desirable. Section §3 outlines a method for obtaining 
uniform moment bounds of multi-dimensional iterated function systems, an 
important area in the field of control theory. Section §4 concerns applications 
in connection to general biochemical reaction networks. 

Finally, we wish to mention that for Markov processes uniform moment 
bounds often imply existence of an invariant probability measure. More 
generally, as the discussion after the proof of Theorem 2.7 shows that a 
uniform moment bound of an appropriate function of the Markov process 
leads to the existence of an invariant probability measure. Thus, the central 
theme of our paper is very much related to the traditional notion of stochastic 
stability. 



2. Proof of Theorem 1.1 

All the analysis hereafter assumes the existence of a probability space 
defined in the hypotheses of Theorem 1.1. 

We start with analogue of [6, Lemma 7]. The proof is just a simple 
application of the following version of Burkholder's inequality [9, §6.3] and 
follows exactly the same steps as in [6, Lemma 7]. For an {J-ij-niartingale 
{M t } taking values in M. d , let {[M] t } denote its scalar quadratic variation 
process (see [10, Chap 2]). 

Lemma 2.1. Let {M t } be an {Ft} -martingale taking values in M. d . Then 
for 1 ^ p < oo, there exists a constant c p > such that 

E[\\M t - M s f\7 s \ sC c p E[([M] t - [M] s )^ 2 \ JF a ], 0^s<t. 

Lemma 2.2. Let {M n } C M. d be an {.F n } -martingale. Assume that for 
some p > 2, there exists a sequence of constants v n such that 

E[||M n+1 - M n \\ p \F n ] sC Vn for all n^O. 

Then there exists a constant c p such that E[||M n — Mfc|| p |J-fc] ^ c p {n — 

Proof. Notice that by Burkholder's inequality there exists a constant c p such 
that 

n 

E[\\M n - M k \\ p \T k ] < c p E[(J2 \\Mj - Mj-xWY 2 ]^] 

j>k 

By Holder's inequality, 



ii ^aif/ 2 ^/ 2 - 1 J2 

i=l i=\ 

Taking a,- = \\Mj — Mj_i|| 2 , it follows that 



p/2 

ay . 



E[\\M n - M k \y\F k ] <: c p (n - kf 2 - 1 J2 EIIIM, - M^ x \Y\F k ] 



j>k 



^ c p {n - kf' 2 - 1 Y. 
j>k 



v r 



U 



Remark 2.3. If in Lemma 2.2 we have the weaker hypothesis: 

E[\\M n+1 - M n \\ p ] ^ v n foralln^O, 

then E[||M n - M k f\ ^ c p (n - kfl^ ^ >k v r 

Lemma 2.4. Let {M n } C R d be an {J" n } -martingale with E[||M || P ] < oo 
and assume that for some p ^ 2, there exists a constant v such that 

E[||M n+ i - M n \\ p ] < v for all n > 0. 

Then for < r < p, there exists a constant 9 = #(E[||Mo|| p ], v,p, r) such that 

E[\\M n \\ r l mn]l>n} ]^e/n^-\ 

Proof. First notice that by Remark 2.3 with v n = u, we have E[||M n — M || p ] ^ 
c p vn p l 2 . Hence, 

(2.1) E[||M n |H < 2 p (E[||M n - Mof] + E[||M || P ]) < B^' 2 

where 9 is a constant depending on E[||M || P ], v and p. Next, notice that 

/■oo 

E[||M n |r l { ||M n ||^n}] = nT(||M n || > n) + / n/^PdlMj > </) dy 

./ n 

/■oo 

< n^E[||M n f] + / rj/ r - 1 -*'E[||M n || p ] dy 

J n 

< E[||M„f) Lr-P + J^ryr-i-P dy \ 



r 



= E[l|M n |n n r ~ p + n r ~ p since r < p, 

\ p — r J 

< #n r - p n p/2 for some 9 > 0, 

where for the last inequality the bound for E[||M„|| P ] from (2.1) is used. □ 

Lemma 2.5. Let {M n } C M d be an {J^j-znartingale with E[||M || P ] < oo 
and assume that for some p > 2, there exists a constant v such that 

E[||M n+1 - M n \\ p \F n } <: v for all n > 0. 

Let t = inf{n > : ||M n || < n}. Then for < r < p, there exists a constant 
6 = fl(E[||M || p ],z/,p,r) such that 



E[||Af n |ri {T>n} ] <0/ 



n p ~ r . 



The proof follows by combining Lemma A.l and Lemma A. 2. The steps 
are essentially similar to that of [6, Theorem 4]. However to make our pre- 
sentation clear, we felt the need to fill in the necessary details for our case 
(see Appendix). 

Lemma 2.6. Let {Z n } be an {.F n }- adapted process taking values in M. d , 
Q C Q measurable, D n a sequence of measurable subsets of IR+ and a a 
stopping time. Call A n = Z n+ i — Z n . Let {7 n } be an {T n }-adapted Re- 
valued process . Suppose that 

o {Z n } is a supermartingale for all 1 ^ n ^ a, that is, E[A n |J r n ] ^ for 
1 ^ n < a, 

o £[||Z p]<oo' 

and either 

(i) E[\\ A n ||P|J r n ] ^ L, for ^ n < oo; 

or 

(ii) E[\\ A n \\ p ] ^ L, for ^ n< oo. 
Assume that on fi n {n < a} 

(iii) Z k -~/ k e D k , forl^k^ n; 

(iv) there exists a constant (3 > such that 7& ^ and ||7k|| ^ fc/5 for 

Then for any < r < p, there exists a constant 9 = 9(E[\\Z \\ P ], L,p, r, (3) 
such that in case of (i) 

Q 

E[||Z n || T ln n{n«T}] < —^.i 
while in the case of (ii) 



E[||-Zn|| r ln n{rKer}J ^ ~ 



p/2-r - 



Proof. As in the proof of [6, Corollary 5], the proof relies on a clever use 
of Doob's decomposition [11, Theorem 5.2.10], [10, p. 74]. By Doob's de- 
composition on each component, there exists a (component- wise) increasing 
predictable process {V n } n ^>i and a martingale {M n } n ^i with Mi = Z\ such 
that 

Z° n = Z aAn = M n -V n , n> 1. 

Note that since {V n } is predictable, E[Z° +1 — Z£|.F n ] = — (V n+ i — V n ). Hence, 

||K+i-Kir^E[||A n || p |j- n ] <L. 
Therefore, 

E[||M n+1 - M n \\ p | J-J ^ 2 p (E[||A n || p | F n ] + ||K+i - Kf) 

sC 2 P+1 L. 

Next, observe that on Q n{n < a}, Z k - ^ k e D k C M+, 1 < fc ^ n. 
Hence by (iii) and (iv), on Q C\ {n < a}, Z k ^ 7fc ^ 0, implying M k ^ 
Vfc + 7fc ^ 7fc ^ 0, for k ^ n. Observe that for x, y G M d , x ^ y ^ implies 
|| x || ^ 1 1 y || . It follows from (iv) that 

tt n{n< a} C {||Af fc || ^ fc^,fc ^n} = {r > n}, 

where r = inf{n > : ||M n //3|| < n}. Moreover, since M n = Z n + V n ^ Z„ 
and on Qq H {n < cr}, Z„ ^ 

||^n|rlf2on{n<er} ^ || M n || r l{ r>n }. 

Now putting M = Z an d using the fact that i?[||Zo|| p ] < oo, we have 
£[||Mip] ^ 2P(E[||Zi - Z p] + £[||Z || P ]) ^ 2 P (L + E[\\Z \\ P ) < oo. The 
assertion now follows by applying Lemma 2.5 to the martingale {M ra //3}„^. 1 . 
The steps are almost exactly the same if we have (ii) instead of (i) , except 
now we apply Lemma 2.4 to {M n /(3}. □ 

Proof of Theorem 1.1. Fix TV ^ 1. Notice that 

E[||G A r(X A r)|| r ] = E[||G A r(X A r)l{x Jv ec}ir] + E[||Gjv(^iv)l{x w ec/\c} IT]- 

Since by (iii), E[||GjvpGv)l{x iV eC'}|| r 1 < b r , we need to prove that 

sup n E[||G n (X n )l {XnGC/ \ c} || r ] < oo. 



To this end, define the last exit time r e of the process {X n } from C up 
to time N by 

r e = m&x{k < N | X k G C}. 

Note that 



G , Ar(X A r)l{X ]v gt/\C}|| I = ^ E M|Gjv(Xjv)l{Xjv6t/\C} || l{r e =fc} 

fc=0 
AT-1 

X)E[ll^(^)|| r l { r.=»}]. 



(2-2) r; 



A;=0 

(fc) 



For any k < N, define the random variables 7„, , by 

n-l 

T^Tf + ^^/^l, ^2, 

with 7o = and 7* = (a, 0, . . . , 0). Define the process \Zn } by 
Z« = (G k+n (X k+n ) + 7 f)l {XfceC} . 

Notice that on the event {r e = k}, X k G C and X k+n G 17 \ C for all 
1 ^ n $C N — k. Hence by the assumptions on the sequences {G n } and {H n }, 
on the event {r e = k} 

(2.3) Z™ - 7 f = Gf fc+n (X fc+n ) G GW(*7 \ C) C R d + , l^n^N-k. 

(2.4) 7 « ^ 0. 

One consequence of the above observation is that on {r e = k} 

(2.5) 

ZW ^ G k+n (X k+n ) > 0, hence \\Z^\\ > \\G k+n (X k+n )\\ for I ^n^N-k. 

Define the stopping time 

a« = inf [j > J Zf - 7 f G M d \ G k+j (U \ C)}. 

It is immediately clear from (2.3) that 

(2.6) {r e = A;} C {cx (fe) > iV - k}. 



Claim: 

• E[Z^ ± - Z { n k) \A k) ] < 0, for 1 ^ n < ^ k \ where A k) = Fk+n, 

• <nin Fril7 (fc) 7 {k) \\ p \ F (fc) l <r no- 

• SU Pfc,n>0 t L|| Z n+l — Z " | l-Ol J < °°i 

• sup fc E[||4 fc) f]<oo. 

Proof of Claim: Suppose k is such that Xk ^ C. Then from the definition, 
Z^ = and the assertions in the claim are trivially satisfied. Next, suppose 

(k) / \ (k) 

that k is such that Xk G C. Then Z\ = Gk+n\Xk+ n ) + Jn ■ Observe that 
for 1 < n < a {k \ we have zf ] - 7 j fc) = G k+j (X k+j ) G G k+j (U \ C) for all 
1 ^ j ^ w. It follows from (ii) that -Xfc +n G U\C, for 1 ^ n < cr( fc ) and we 
have using (1.1) 

E[Z n+ i - Z\ | 7\ '] = E[G k+n+ i(X k+n+1 ) - Gk+n(Xk+n) | Fn ] + Hk+n{X k+ n) 

<0. 

where 7J = 7-fc +n . Moreover for ^ n < oo, 

E[||45i - 4 fc) |r I ^] < 2P (E[||G fc+n+ i(**+n + i) - G,+„(X fc+n )f | Jf)] + 6P) 

^2 p (L + #>). 

Lastly, by (iii), E[||zff] = E[||G fc (X fc )||l {XfceC} ] < sup n su Pa;eC ||G n (x)f < 
If. Hence the claim follows. 

Furthermore, on {r e = k} n {n < a^}, ZJ fc) - 7 f } G G fc+i (t/ \ C) and 

X fc+ j G 17 \ C for all 1 ^ j ^ n implying that 7- ' ^ for 1 ^ j ^ n + 1. 

Observe that that from the definition of 7« , ||7n || ^ n(3 for some constant 
/3. To see this use the fact that there exist constants fli and /3 2 such that 
All^ll ^ IMIi ^ /^IMI f° r all x G M d , where H-^ denotes the standard t\- 
norm on M. d . Next noting that \\x + y\\\ = \\x\\i + ||y||i for x, y G IR+ we 
have 







2 




n-l 

i75 fc) n+x;ii^*+i(^*+i) 



10 



>£(o + o(n-l)) 

^ /3n, for some constant depending on a and dimension d. 

Now Lemma 2.6 gives that there exists a constant (depending on p, a, b, d, L, r) 
such that for < r < p, 



(2.7) 



7(k) nr-, 

'JV-fcll 1 {r e =fc}n{cr( fe )>Ar-fc} 



< 







(N -k)P~ r ' 
Finally, from (2.5) and (2.6) it follows that for k = 0, . . . , N - 1, 

||G'Af(^Ar)|ri{r e =fe} = ||Gfc + jV_fc(Xfc + (JV-fc))|| r l{Te=fc} ^ ||^Ar_fciri{r e =fc}n{<r( fc )>A r -fc}- 

Hence by Lemma 2.6, we have 

r_i Ar_i 





JV-1 



iV-1 



E[||G^(X W )1 



{XivG!7\C} 



r ] =^[\\GN(X N Wl {Te=k} ] ^Yl 



k=0 fc=0 

^ #CG° — t) < oo, for < r < p — 1. 



(TV - fc)?- 



where £ denotes the Riemann zeta function. 

If in (v), instead of (1.2) we have (1.3), then by Lemma 2.6 instead of 
(2.7), we have 



A k ) \\r-t 

J N-k\\ L {r e =k}n{a( k )>N-k} 



and the rest of the proof stays the same. 



< 



9 



(N - k)p/ 



2-r ' 



□ 



Let {/C^ | a = 1, . . . , 2 d } denote the collection of all standard orthants 
of M. d , i.e., the sets {z G M. d \ Zi ^ or Zi ^ for each i = 1, . . . , d}. Recall 
that if /C is a non-empty positive convex cone in M. d , then the conic (partial) 
order ^ induced by K is defined by x ^ y if y — x G /C. For x, y G M. d and 
an orthant K d a we define x ^ Q y if y — x G /C„. To keep consistency with our 
earlier notation, when the orthant in consideration is M+, we write x ^ y, 
whenever y — x G M+. We note that in Theorem 1.1, the orthant R+ plays 
no special role, that is, the statement of the theorem is true if we replace M^ 
by any orthant K d a . More precisely, 



11 



Theorem 2.7. Let (&,, J 7 , {.F re }, P) be a filtered probability space, U a 
complete and separable metric space and C C U measurable. Let the orthant 
K d a be defined by 

JC d a = {xeR d \x n ^0,...x k ^0, Xj ^0,j^i h ...,ii}. 

Let {G n : U — > R d } and {H n : U — » M. d } be sequences of measurable functions 
satisfying 

(i) for every n, G n , H n : U \ C ->• /C£; 

(ii) for every n, G~ l G n (U \C) = U\C; 

(iii) there exist constants a,b > 0, such that a ^ inf n inf a^c 1 ||if n (x)||, 
sup n swp x<jLC \\H n (x)\\ ^ b and sup n sup xeC \\G n (x)\\ ^ b. 

Let {X n } be a sequence of {J-'n}- adapted U-valued random variables. Assume 
that Xq G C and the following two conditions hold: 

(iv) for all n ^ 0, 

(2.8) E[G n+1 (X n+1 ) - G n (X n ) | F n ] ^ a -H n (X n ) on {X n £ C}; 

(v) there exist constants L > and p > 2 such that for all n ^ 

(2.9) E[||G„ +1 (X„ +1 ) - G n (X n )\\> | Jn] ^ L. 

Then for any < r < p — 1, there exists a constant n = rj(p, a, b, d, L, r) such 
that 

sup E[||G n (X n )H r ] ^n. 



If instead of (2.9) we have 

(2.10) E[\\G n+1 (X n+1 ) - G n (X n )\\P] ^ L, 

then for any < r < p/2 — 1, there exists a constant n = rj(p, a, b, d, L, r) 
such that 

sup E[||G n pQ|| r ] ^n. 
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Proof. The proof is exactly the same as that of Theorem 1.1 since all the 
steps remain valid if we replace Mi by Kr a . Alternatively, we can derive 
Theorem 2.7 from Theorem 1.1 by 'rotating' the orthant K. d to K+. 
For x G M d , define the operator 5 a on M d by the following action on x: 

alter the sign of x^, . . . ,Xi p and keep the remaining co-ordinates unchanged. 

Note that S a is a self adjoint and unitary operator and 8 a {lC^) = K+. Define 
the sequences {G%} and {H"} by 

G a n {x) = 5 a G n (x), H%(x) = 5 a H n (x) 

Then (i) and (ii) imply (i) and (ii) of Theorem 1.1 for the sequences {G®} 
and {H%}. Also, a ^ inf n inf^ c ||P^(x)||, sup n sup^ c \\H*(x)\\ < b and 
sup n sup xeC . ||G"(x)|| ^ b. Finally, (2.8) and (2.9) imply that the sequence of 
processes {X n } satisfies 

. E[G a n+l (X n+l ) - G«{X n ) | F n ] ^ -H«(X n ) on {X n $ C}, 

. E[\\G a n+1 (X n+1 )-G«(X n )r\F n ] ^L. 

Consequently, Theorem 1.1 says that there exists an t] > such that for all 
n G Z+, 

E[|raX B )|n = E[||G B (X B )|n<,7 

n 

Existence of invariant probability measures for Markov chains 

Let U be a complete and separable metric space. Let {X n } be a [/-valued 
Markov process with transition kernel P : U x 95(f) — > [0, 1], where 95(f) 
denotes the Borel cr-algebra on U. For g : U — )■ R d , define Pg : U — > M d 
by Pg(x) = $jjP(x, dy)g(y)_ = E\g(Xi)\X Q = x] = E x [g(X 1 )]. Suppose that 
C C U is measurable and P re denotes the closed Euclidean ball of radius k 
centered at the origin in M d . Assume that 

(i) P is (weak) Feller, i.e., if / : U — > M. is a continuous and bounded 
function, then Pf is continuous and bounded; 

(ii) there exist a measurable map G : U — > M d , a measurable function 
H : U — > M d and constants a, b > such that 



there exists a G {1, . . . , 2 d } such that G,H : U\C ->■ /C 

13 



a ' 



o G- 1 G(C/\C) = U\C; 

o G^ 1 (B K ) = {x G IR m : ||G(x)|| ^ k} is compact for every k > 0; 

o sup^cllG^)!! < oo; 

o a^ \\H(x)\\ ^b, for all x (£ C; 

(iii) for all x G t/ \ C 

(2.11) PG{x) - G{x) ^ a -H (x); 

(iv) there exist constants L > and p > 2 such that for all x G £/ 
P(||G(-) - G(x)|r)(x) = ExOlGM - G(x)in < L. 

Then {X n } has an invariant probability measure. To see this, first observe 
that an application of Theorem 2.7 gives sup n E x [||G(X n )|| r ] < oo for all 
< r < p — 1 and x G G. Fix < r < p — 1 and x G G. Let e > and let «; 
be such that sup n E x [||G(X n )|| r ]/fi; < e. Notice that 

P x (X n i G-\B K )) = P.dlG^)!! > k) < supE r [||G?(X n )|| r ]/K < e. 

n 

Since G _1 (5 K ) is compact, it follows that {P n (x, •)} is tight. Define the 
Cesaro sum P^ by 

n-l 

k. 



p{n) ^-) = l -H p 



n ^— ' 

fc=0 

It is immediately clear that the sequence of probability measures {p( n \x, •)} 
is tight and hence relatively compact. Let \x be a probability measure on [/ 
which is a limit point of {P^ n ^(x, •)}. Then the Krylov-Bogoliubov theorem 
([2, Proposition 7.2.2], [12, Theorem 3.1.1]) shows that fi is invariant. 

3. Uniform moment bounds for discrete-time iterated function sys- 
tems 

Consider a discrete-time Markov process {Z t } te z + = {(%t,yt)}t£Z + taking 
values in Mi x V, where V is a countable set, defined by the following rules: 



(IFS1) for each i G V there exists a measurable mapping /(•, i) : 
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l d + ; 



(IFS2) there exists a measurable map P : Ri xV xV — y [0, 1] such that for 
each (x, y) G Ri x "P the function P(x, y, •) is a transition probability; 

(IFS3) at time t = n, given the state (x n ,y n ) = (x,y), 

o first y n +i is selected randomly according to a time-homogenous but 
^-dependent transition kernel P x (y, •) = P(x,y, •), and 

o given y n+1 , we set x n+ i = f(x,y n+1 ). 

Observe that neither of the process {x t }tei+ or {yt}t& + is Markovian on its 
own. The transition kernel of the process {Z t }t^z + stands as 

P(Z t+1 = (x',y') | Z t = (x,y)) = P x (y,y')5 f(X:y/) (x'), 

where 5 is the Dirac measure. The stochastic system 

(3.1) x t+ i = f(x t , yt+i), (xo, yo) ER d + xV given, 

derived from the process {Z t } t£ z + = {(%t, Ut)}tez + constructed above is 
known as an iterated function system with place dependent probabilities [13]. 
These systems are generally employed in the synthesis of fractals, modeling 
biological phenomena, etc [14]. 

Iterated function systems are important objects of study in control theory, 
where they are known by the name discrete-time stochastic hybrid systems 
[8, 15, 16, 17]. There is a considerable literature addressing classical weak 
stability questions concerning the existence and uniqueness of invariant mea- 
sures of iterated function systems, see e.g., [18, 19, 20, 21, 22] and the refer- 
ences therein. The arguments in these articles predominantly revolve around 
average contractivity conditions of the iterated function system, and conti- 
nuity of the probability transitions. Stronger stability notions such as exis- 
tence of moments of sufficiently high order mostly involve Foster-Lyapunov 
drift conditions, which in turn work best under the average contractivity 
assumption. Although there have been efforts to relax average contractiv- 
ity conditions in conjunction with Foster-Lyapunov drift conditions, see e.g., 
[23], generally the assertions consist of sub-geometric rates of convergence of 
Markov processes to their invariant measures; moreover, such techniques do 
not extend directly to moment bounds. Furthermore, in real-world control 
applications the average contractivity property generally translates to requir- 
ing unbounded control actions, which is hardly ever possible to guarantee. In 
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this section we give conditions for uniform L r (P)-boundedness of the process 
{x t }t£Z+ generated by (3.1) in the absence of average contractivity. To this 
end, we further assume that 

(IFS4) there exists a constant L > such that 

\\x — f(x,i)\\ ^ L for every (x,i) G K+ x V. 

Remark 3.1. Observe that the existence of a uniform bound on the jumps 
hypothesized in condition (IFS4) above implies that an "average contractiv- 
ity" condition is impossible to satisfy without transforming coordinates. The 
condition (IFS4) holds for a large class of realistic nonlinear control systems, 
especially under bounded control actions. 

We have the following: 

Proposition 3.2. Consider the system (3.1) and suppose that the con- 
ditions (IFS1), (IFS2), (IFS3), and (IFS4) hold. In addition, suppose that 
there exist a measurable bounded set C C M.+ and a vector a G R with a > 
such that 



J2 Px(y, V')f(x, y') - x < -a— for (x, y) G (Rj \C)xV. 
y'&P 



Then the process {x t }t£Z + is L r (P)-bounded for every r > 0. 

Proof. Let {J r t }t& + be the natural filtration generated by the process {Z t }tez+ . 
For any given p > 2, we see at once that the condition (IFS4) implies that 

(3.2) E[||x m - x t \\ p | Ft] < L p for all t G Z+; 

therefore, condition 1.2 of Theorem 1.1 holds. Moreover, 

E[x m -x t \j z t ]= I ^2 P *t(yti y')df(x t ,y')( x ') id(x') dx' - x t 

= ^2 p x t (yt,y f )f(x t ,y') - x t 

y'GV 

< -a-^-r on {x t ER d + \C} 

in view of our hypotheses; therefore, condition 1.1 of Theorem 1.1 holds with 
G n (x) = x and H n (x) = o-nfir- We conclude that the process {x t }tez + is 
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L r (P) bounded for all < r < p — 1. Furthermore, since the bound on the 
right-hand side of (3.2) is finite for all p > 0, Theorem 1.1 also implies that 
{xt}tez + is L r (P)-bounded for every r > 0. The assertion follows. □ 

4. Connection to biochemical reaction systems 

A biochemical reaction system involves multiple chemical reactions and 
several species. In general, chemical reactions in single cells occur far from 
thermodynamic equilibrium and the number of molecules of chemical species 
is often low [24, 25]. Recent advances in real-time single cell imaging, micro- 
fluidic techniques and synthetic biology have testified to the random nature 
of gene expression and protein abundance in single cells [26, 27]. Thus a 
stochastic description of chemical reactions is often mandatory to analyze 
the behavior of the system. The dynamics of the system is typically mod- 
eled by a continuous-time Markov chain (CTMC) with the state being the 
number of molecules of each species. [28] is a good reference for a review of 
the tools of Markov processes used in the reaction network systems. Ana- 
lyzing stability of stochastically modeled biochemical reaction systems (e.g, 
gene regulatory networks) in particular, questions dealing with existence of 
invariant probability measures, moment bounds are important both for ex- 
perimental and theoretical purpose [29, 30]. The goal of this section is to 
outline a method to investigate these kind of stability questions for biochem- 
ical reaction networks. 

Consider a biochemical reaction system consisting of n species and v 
reactions, and let X(t) denote the state of the system at time t in Z™. If 
the k-th reaction occurs at time t, then the system is updated as X(t) = 
X{t— )+i^k~ u ki wnere X{t— ) denotes the state of the system just before time 
t, and v^-i^t *= ^+ represent the vector of number of molecules consumed 
and created in one occurrence of reaction k, respectively. For convenience, 
let v k = v^ — v^. The evolution of the process X is modeled by 

P[X(t + At) = x + u k \X(t) = x] = a k {x)At + o(At). 

The quantity a k is usually called the propensity of the reaction k in the 
chemical literature, and its expression is often calculated by using the law of 
mass action [31, 32]. The generator matrix or the Q-matrix of the CTMC X 
is given by q XyX+Uk = ak{x). The CTMC X will have an invariant measure n 
if itQ = 0. 
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Let B p be the standard open ball of radius p centered at in M. n . Assume 
that there exist a function H : 7D\ —$■ Wt and a constant p > such that 

(BRS1) there exist constants a, b > such that a ^ ||if(x)|| ^ b, for x G 

V 

(BRS2) F(x) = ^ a k {x)v k ^ -If (x)A(x) for x£Z B + \ B p , where A(x) = 
fc=i 

(BRS3) Fp(x) = 2_, °fc( a 0ll r/ fc|| p ^ -^^4(x) for some constants p > 2 and L > 
fc=i 
0. 

Possible examples of if include constant vector with positive entries, if (a;) = 
(ax + f3)/\\ax + f3\\,a > 0,(3 ^ 0, etc. Let {Y n } be the jump chain corre- 
sponding to the CTMC X. That is, putting tq = 0, we define inductively 

r n+1 = inf{£ > T n | X(t) ^ X(r n )}. 

Notice that r n denotes the n-th jump time of the CTMC X. Define Y n = 
X(r n ). {Y n } is a discrete-time Markov chain and is often called the jump 
chain or the skeleton chain corresponding to the CTMC X. Now (BRS1) and 
(BRS2) imply sup^ 7^ [||l^T.|| r ] < oo for < r < p — 1. To see this, we first 
obtain the transition matrix of the Markov chain { Y n } from the Q-matrix of 
X (see e.g., [33, p. 108]). Specifically, 



P(Y n+1 = x + v k \Y n = x) 



a k (x)/A(x), i£A(x)^0 


0, if A{x) = 0. 


(o, HA(x)^0 


1l, HA(x) = 0. 



P(Y n+1 = x\Y n = x) 



In most biochemical reaction systems A(x) > for all x G Z™ or at least 
outside a compact set of Z™ . Now for all i62"\ B p , by (BRS2) 

V 

E[Y n+l -Y n \Y n = x]=J2 v k a k (x)/A(x) < -H(x) 



Moreover by (BRS3), for all ieZ" 



V 



E[\\Y n+1 - Y n \\ p \Y n = x]=}_^ \\v k fa k {x)/A{x) ^ L 

fc=i 

Thus (BRS2) and (BRS3) imply (1.1) and (1.2) of Theorem 1.1 for the 
Markov chain {Y n } with G n (x) = x, and consequently, sup n £'[||F n || r ] < oo 
for < r < p — 1. Now, the discussion after the proof of Theorem 2.7 
shows that {Y n } has an invariant probability measure A. Consequently, it 
follows from [33, Theorem 3.5.1] that if A(x) > for all x G Z+, then ir(x) = 
X(x)/A(x) is an invariant measure for the CTMC X. If mi xG %n A(x) > 0, 
then the CTMC X has an invariant probability measure. Of course, if we 
are just interested in the existence of an invariant probability measure and 
(BRS2), (BRS3) do not hold, then the discussion after the proof of Theorem 
2.7 can be employed to look for a suitable G. 

Appendix 

Lemma A.l. Let {M n } and r be as in Lemma 2.5 and assume that for 
some p > 0, there exists a constant v such that 

E[||M n+1 - M n \\ p \F n ] ^ v for all n ^ 0. 

For k > 0, let S k = inf{j > : ||AfJ ^ fc/3}. Let T k = inf{j > : 
||Mj + i — Mj\\ ^ fc/3}. Then there exists a constant 9' such that 

E[||M B |ri {T>B} l {fc<rn} ] ^ 



n p-r 

Proof. For notational convenience, put S n = S and T n = T. First, notice 
that from the definitions, {r > n} C {||M n || ^ n ^ w/3} C {S 1 ^ n}. Also, 
on the event {S ^ T}, ||Ms — Ms_i|| ^ n/3 and since ||M,sr_i|| ^ n/3, we 
have ||M S || < 2n/3 on {5 ^ T}. Now 

E[||M n || r l{ T>n }l{5^ T }] = E[||M n || T 'l{ T>n }l{5 s ; n }l{5 s ; T }] 

— E[E[||A / / n || r l{ T>n }l{ S ^ n }l{s^T}|^ : 5]] 
(A.l) = E[[l {s ^ n} l {s<T} E[\\M n \\ r l {T>n} \T s }} 

By the optional sampling theorem, Lemma 2.2 gives 

E[||M n - M SAn \\P\ F SAn ] < c p u(n -SA nfl\ 
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Therefore, on the event {S ^ n}, 

E{\\M n -M s \\ p \F s }^c p vn p / 2 . 
Now, for a non-negative random variable Z 

/■oo 

(A.2) E[Z r l {z>u} \Q] = u r P(Z > u\g) + / rf~ l P{Z > y\Q) dy, 

J u 

where Q is a sub er-algebra of J 7 and Q' G J 7 . Hence 

E[||M n |ri {T>n} |Js] = E[||M n |ri { || Mn ||^ n} |J- s ] 

/>oo 

= n r P({||M n || > n}\F s ) + / ry r - x P({\\M n \\ > y}\F s ) dy 

J n 

<:n r P({\\M n -M s \\ >n/3}\F 8 ) 

ry^ l P{{\\M n - M s \\ > (y - 2n/3)}\JF s ) dy, on {5 < T} 

/■oo 

+ / ry r - 1 E[\\M n -M s \\f\JF s ](y-3n/^- p dy, on {S^T} 

J n 

^ 9 1 n r -PE[\\M n - M s \\ p \Fs], on {S ^ T} 
<: c p vQ 1 n r - pl2 '; on {S ^ T} n {S ^ n}. 

Thus (A.l) implies that 

E[||M n |ri {r>n} l {s<T} ] ^ c^0 1 r i '-^ 2 E[l { 5 <n} l { 5<T}] 
(A.3) sC c p v6 1 n r - p/2 P(S < n). 

Notice that 

P(5 < n) < P(||M 5An || ^ n/3) ^ 3 p E\\M SAn \\ p /n p 

^ 3 p E||M n || p /?7, p , since {||M n || p } is a submartingale 

<y ^!, by(21) 

Plugging the bound for P(5 ^ n) in (A.3) we are done. □ 
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Lemma A. 2. Let {M n } and r be as in Lemma 2.5 and assume that for 
some p > 2, there exists a constant v such that 

E[||M n+ i - M n \\ p \T n ] sC v for all n ^ 0. 

Let S k and T k be defined as in Lemma A.l. Then there exists a constant 6" 
such that 

E[||M n iri {r>n} i {5n>r „ } ] < j^. 

Proof. As before, we denote S n = S and T n = T. Again since {r > n} C 

n-l 

E[||M n || r l{ T>re }l{ T<( 5}] = 2 , E[||M ra || r l{ T>n }l{g > fc}l{r=fc}] 

fc=0 
n-l 

^3^E[(||M fc |r + ||M fe+1 -M fc |r 

fc=0 

+ \\M n — M fc+ i|| f )l{ r> „}l{s'>A ; }l{T=fc}] 
n-l 

^3 r ^(J + // + ///). 

fc=0 

First notice that since on {S > k}, \\M k \\ ^ n/3 and r > n implies r > k, 
for k ^ n, we have 

77 

/^E[i {5>fc} i {T>fc} (-rp(T = ^)] 

Ti 77, 

< E[l {5>fc} l {r>fc} (-)T(||M fe+1 - M k \\ ^ -\T k )]. 

Notice that 

(A.4) P(||M fc+1 - M k \\ > n/3\T k ) ^ 3 p n~ p E[\\M k+1 - M k \\P\T k ] < 3 p n~ p u. 

It follows that / ^ 3 p - r n r - p uP(T > k). 
Next, observe that 

II ^ E[l{ T>fc }E[||M fc+ i - M k \\' l{\\M k+1 ~M k \\>n/3}\F k \\. 

Then from (A. 2) we have 

(77\ r n 

-J P(||M fc+ i - Mfc|| ^ -l-Ffc) 
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POO 

+ / ry r ~ l P(\\M k+1 - M k \\ ^ y\T k ) dy 

h 

^E{\\M k+1 -M k \\ p \F k ](3 p - r n r - p 
n/ r " 1_p dy) 



'n/3 

^ 02n r ~ p , for some #2- 



Hence, 77 ^ e 2 n T ~ p P{r > k). 
Finally, 



(A.5) /// ^ E[l {r>fc} l {r=fc} E[||M n - Affc+xin J" fc+1 ], 

as by the definition {T = A;} is J-fc + i-measurable. Notice that by Lemma 2.2 
E[||M n - Mj fc+1 || 3, |J r fc+ i] < c p v(n - k - l) p/2 , for k < n. Now since r < p, it 
follows that 

E[||M n - M fc+1 || r | J" fc+1 ] ^ E[||M n , - M fc+1 ||P| F k+l } r ' p 

^ (c p z/) r/p (n - fc - l) r/2 , for jfe < n 
^ (c p z/) r/p n r/2 , for jfe < n. 
Putting this in (A.5) we have for k < n 

III ^ {c p vyl p rf/ 2 E[l {T>k} l {T=k} } 

= (c p u) r / p n r / 2 E[l {T>k} P(T = k\T k )] 
< (c p v) r / p n r / 2 E[l {T>k} P(\\M k+l - M k \\ ^ n/3\T k )] 
^ (c p v) r/p 3 p n r/2 - p P(T >k), by (A.4). 
Now for k ^ 1 

P(r>k)<:P(\\M k \\^k)^E[\\M k \\ p }k- p 

^9 k~ p / 2 , by (2.1). 

Since p > 2, it follows that for some #3, 

n-1 

A;p/ 2 - 
^9 3 (l + 6 ((p/2))n r - p , 
where ( denotes the Riemann-zeta function. 



E[||M„|ri {T>n} l {T<s} ] < 6 3 n r - p (l + J2 

k=i 



D 
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